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THE NEED 

There is a growing consensus that traditional methods, such as standardized testing, criterion-referenced tests, 
and teacher-constructed tests fail to measure important learning outcomes (Shepard, 1989; Anderson & Bachor, 
1998; Shavelson & Ruiz-Primo, 2000; Koker, 2001; Davies, 2001). Such tests provide little to indicate either the 
level at which a student understands or the quality of individual thinking (Nickerson, 1989; Slack, 1993; 
Raychaudhuri, 1998; Lee, 2002). They emphasize homogenized recall of memorized factual knowledge and 
procedures rather than unique, and highly differentiated reflection. Because external criteria, they typically 
emphasize standards which can be applied to typical students. Changing the way we assess will inevitable 
change how teachers teach and how students learn. Today new ways of thinking about learning call for new 
ways for monitoring learning. 

Reform in school assessment builds from the vision that assessment can become the bridge for instructional 
activity, accountability, and teacher development. Romberg (1995) stated that if assessment results are used by 
the learners or teachers, then the assessment tools must be available in the classroom on a regular basis, weaving 
together instruction and assessment (p. 29). The content of tests influences teaching and learning processes. 
Teachers often "teach to the test" rather than emphasizing underlying concepts. 

Skills are thought in the manner measured on tests rather than how they are used in everyday contexts. When 
tests require the recall of memorized information, students develop memorization strategies that tend to de- 
contextualize their knowledge, promoting compliant cognition (McCaslin & Good, 1992; Pettig, 2000; Dolan & 
Hall, 2001). In order to become capable, learners need experience in solving real problems and understanding 
complex tasks (Duffy, 1997; Linn et al., 2000). Shepard (1989) stated that assessments need to approximate 
real-life tasks and to reflect multiple perspectives and diversity-versus-singularity of problem solutions. 

Another problem with traditional testing is that it tends to emphasize evaluation, or classification, as a primary 
goal (Hart, 1994; Wilson, 1995; Ayala et al., 2002; Yin et al., 2004). Since a primary goal of education is to 
promote students' thoughtfulness, the basic of concept of testing needs to change, not just the structure of the 
tests (Brown, 1989; Koretz, 1998; Sizer, 2001). Also, McLellan (1993) pointed out that assessment needs to be 
dynamic, and reflect every-emerging samples of the learner's progress. As a consequently, traditional testing 
strategies are often counter-productive for the solving of real-world problems (Collins, 1990; Yin & Shavelson, 
2004). 

Choi & Hannafm (1995) and Reese (2003) stressed that in order to be useful in promoting higher thinking skills, 
testing needs to shift from domain-referenced evaluations to student-center assessments. Student-centered 
assessment emphasizes the ability to diagnose and manage cognitive growth rather than to evaluate student 
achievement. They said that since assessment in situated learning environments emphasizes cognitive and 
learning processes, improvement of learning strategies, and higher-order thinking skills, assessment alternatives 
typically require varied evidence (Pettig, 2000). As a programmatic change is occurring, there is a need to align 
student assessment practices with curricular aims, instructional practice and performance standards (Black 
&Wiliam, 1998; Their & Daviss, 2001). 

The development of problem solving involves students' efforts to overcome obstacles and attain goals (Stecher, 
1998). It involves the orchestration of a large number of other processes toward this end (Siegel & Thier, 2002). 
How well students encode to form mental modes are among the key determinants of their success on many 
problems. As Siegler (1991) stated that their success also depends critically on the ability to integrate general 
and specific knowledge, and on their selection of the right process in the right situation. Choi and Hannafm 
(1995) pointed out that constructivists will have to develop ways of expressing what is to be accomplished that 
do not constrain learning outcomes as they feel specific objectives would. Without some idea regarding student 
outcomes, evaluation would be an empty exercise (Pisha & Coyne, 2001; Sizer, 2001; Reese, 2003). 
LITERATURE 

Traditional assessment is the process of gathering information about students- what they know and can do. In 
fact, assessment data simply mirrors what is going on in the classroom. This information become meaningful 
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only when we decide that it reflect something that we value, such as how well a student has mastered long 
division. Authentic assessment emphasizes the development of assessment tools that more accurately mirror and 
measure what we value in education (Hart, 1994). An assessment is authentic when it involves students in tasks 
that are worthwhile, significant, and meaningful (Siegel et al., 2002). The computer can provide a further 
perspective on the learner. It can effectively track the process of learning as well as a learner's response to 
feedback. It can also “simulate realistic situations in the classroom” (Lajoie, 1995, p.28). 

The computer provides opportunities for assessing the dynamic nature of problem solving and opportunities to 
systematically vary the instructional environment on feedback dimensions and observe the effects on learning 
outcomes (Rose & Meyer, 2002). Computers make possible the dynamic assessment of relevant criteria (Newby 
et al., 1996). Most of the evaluation procedures involve having evaluators use a rating form (e.g., Litchfield, 
1992; Tolhurst, 1992; Voogt, 1990; Robert & Wilson, 1998; Thier et al., 1999; Wilson & Sloane, 2000) to 
evaluate each of a variety of features of a piece of educational software for classrooms. An overview of the 
components, functions and limitations of the human cognitive system provides a framework for understanding 
why some educational software that "looks good" fail to produce positive student’s outcomes (Siegel et al., 
2002). Unfortunately, nearly all software evaluation systems are heavily weighted on computer-related 
dimensions of error-handling and aesthetic considerations, such as the quality of screen displays, sound, touch, 
and content related issues of scope, sequence, and accuracy. Although important, these characteristics do not 
address the consistency of our knowledge with how students learn (Charleston, Villagomez, & Shaffer, 1989; 
Lounge et al, 1986; Wilson et al., 1996; Thier et al., 1999; Thier & Daviss, 2001). 

Reiser and Kegelmann (1994) stated that there is no evaluation methodology which is equally applicable to the 
service of the strategic intentions of administrators, the empirical requirements of academics and the needs of 
classroom practitioners for information in support of tactical decision-making. Then they pointed out that it is 
important that software evaluation organizations incorporate examinations into their software evaluation 
processes. By doing so, these organizations will take a significant step forward toward accomplishing their 
primary mission—assisting educators in identifying software that will truly enhance student learning. In light of 
these facts, what can organizations responsible for the evaluation of software do to improve their evaluation 
methods? They also stated that in order to overcome the problems associated with subjective evaluations, those 
who have critiqued software evaluations techniques often suggest that an examination of student use should be 
an important part of the process. Most researchers suggest that evaluators collect attitude data from the students 
who have worked through the program (Schuecker & Shuell, 1989; Jolicoeur & Berget, 1989, Barab et al., 
1996; Davies, 2001; Yin et al., 2004). Scholars point out that portfolio; protocol analysis (think a loud) 
performance assessment and concept maps are the most popular alternative, authentic and meaningful methods 
of the assessment of situated learning (Collins et al. 1993; Wolf, 1989, 1998; Wolf et al. 1991; Lajoie, 1995; 
Ayala et al., 2002; Yin et al., 2004). Wilson & Sloana (2000) said that a clear vision of the overall framework 
can be constructed with a coherent authentic assessment system. 

FROM SOFTWARE ASSESSMENT TO SOFTWARE MAPPING 

Science and mathematics educators are enjoined to assess what students know and come to understand through 
knowledge representation (Harlen, 1985; Gentner & Markman, 1997). Representing such knowledge can be 
accomplished by the use of technology that provides, not only a vehicle for their storage, display, and active 
presentation but also a "malleable and interchangeable" electronic forum for products of the mind. Over the past 
decade, research on human cognition has revealed insights into the mental processes involved in learning, 
remembering, reasoning, and problem solving. These findings also have implications for the design and 
evaluation of instructional software (Rose & Meyer, 2000a, 2000b and 2002). 

The process of learning with computers is influenced by the ability of the medium to dynamically represent 
formal constructs and instantiate procedural relationships under the learner’s control (Kozma, 1991). The ability 
to control the flow of action and events in the software allows the instructor to tailor a lesson to the specific 
needs of the learner. Menu systems, design components (such as diagnostic routines, unit planning assistance, 
flexible lesson sequencing, multi-level lesson sequences, flexibility to easily branch, random use and so on), 
interaction with user, and management services are the components of functional design. Thus, software design 
must allow for smooth and rapid exit from one menu to another with minimum effort. Menus should provide a 
non-destructive means to cancel or abort the action selected to allow the learner to escape from an unintended 
choice. It is only in this concern over components of functional design that menu, sub-menu, and tool variety 
become explicit. 
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One study that prompted further thinking of software mapping was that of Schuerman and Peck (1991) who 
studied the effects of pull-down menu (PDM), returning to pull-down menus or the sub-menu (RSM) options, 
and return to main menu (RMM) options as exercised by users of the graphic-user interface. PDMs did not 
necessarily encourage random, as opposed to sequential, access of lesson activities. The RSM condition 
promoted significant grouping of options. The PDM condition did not produce weaker groupings than RMM 
condition. PDMs yielded significantly higher menu inspection; i.e. looked but did not "leap." Ergonomics of 
the PDM does not diminish deliberation on the part of the user. The RSM option facilitates return to sub-menu 
item 80% of the time whereas RMM does not. And the "compact menu system provided by pull down sub¬ 
menus and a menu bar offered full functionality of a two-level, tree-structured menu system without the 
formality of full-screen static menus (pp. 97- 98). 

It has been stated that "courseware is often instructionally sound but fails because it lacks the touches of the 
creative mind-spontaneity, humor, variety, and pizzazz” (Kearsley, 1985, p. 217). Effective software design 
permits curriculum content to be delivered in a manner most appropriate for the target group of learners 
(Hernandez & Reese, 2004). Appropriate software design allows the learner to obtain the maximum benefits 
from the material being presented (Fernandez & Body, 1997; Fisher, 2000). The amount of effort required of the 
learner to interact with the software will be rewarded by a sense of satisfaction. Effective learner's interaction 
with the software will result in immediate delivery of desired, expected, and relevant information from the 
software (Barab et al., 1996; Draney & Wilson, 1997, Kurtz et al., 2001). The concentration of the learner 
should be focused on the curriculum material and content and not on the process required to interact with the 
software. Interaction with the software needs to be as natural and intuitive as possible (Charleston, Villagomez, 
and Shaffer, 1989; Kumar, 1994). Interaction between teacher, student and computer may lead to significant 
changes in the processes rather than the measurable products of education (Barab et al., 1998; Shavelson & 
Ruiz-Primo, 2000; Reese & Hergert, 2004). Evaluation must therefore be sensitive to classroom process, and 
must be capable of providing information about these processes. 

Without a doubt the graphic-user interface or usability engineering has achieved a favorable status in 
educational computing. We now thought of having a vehicle, with such an interface, to explore the behavioral 
content (actions) of students as they "navigate" on screen and through the software. Before developing software¬ 
teaching training programs, we need to have a more through understanding of what particular aspects of 'good 
practice' with computers lead to what specific gains in the quality of pupil learning. It is at this point that 
software mapping is proposed to match the curriculum with student behavior. 

OVERVIEW SOFTWARE MAPPING 

Software mapping is simple but proposed to be a dynamic method that facilitates access to general software 
program parameters and engenders a knowledge of powerful, but embedded and "deep-rooted" options allowing 
for increased inter-rater reliability in documenting or replicating richly constructed student responses. Rich and 
diverse student constructions can only be mediated by rich and diverse curriculum activities. Designing a 
sequenced set of student activities which use program menus, submenus, and palettes and constructing probable 
maps associated with each activity reveals divergence in thinking on the part of the student. 

The purpose of the software mapping is to delineate a method for software menu, tool, and palette use in the 
construction of elementary school science and mathematics curriculum activities. With this method, software 
'maps' were created for traversing science and math curriculum problems and activities using software. The other 
purpose of the software mapping efforts is to delineate a method for student assessment in the classroom. 
Software mapping is an approach to assessment documenting behavioral content in computer interaction. 
Examples of mapped problems are presented, analyzed, and discussed in this paper. These examples are derived 
from curriculum handbooks received from Apple Corporation through efforts of the National Council of 
Teachers of Mathematics (NCTM) and the Association for the Education of Teachers of Science (AETS) (Apple 
Computer, Inc., 1992; 1993). 

THE SOFTWARE MAPPING METHOD WITH KID PIX 

A unit of study focused on investigation entitled "Patterns, Patterns Everywhere, and Not a Drop of Ink" is to be 
implemented using documentation provided for the program Kid Pix from Broderbund Software (Apple, 1993, 
pp. 119-50). Kid Pix and Kid Pix Companion are authored by Craig Hickman (1991, 1992). The mathematical 
concepts and methods used are constructed for students in grades 2-4. NCTM Standards addressed are 
Mathematics as communication, Mathematics as reasoning Geometry and spatial sense, Measurement, and 
Patterns and relationships. Specifically "the explorations will enable students to: 
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• Describe, model, draw, and classify shapes 

• Investigate and predict the results of combining subdividing and changing shapes. 

• Relate geometric ideas to the concepts of numbers and measurement. 

• Recognize and appreciate geometry in their world. 

• Communicate their understanding of geometry both verbally and pictorially” 

(Apple,1993, p.120). 

Classroom management is highlighted early in the unit. Equipment requirements, software preparation, helpful 
hints, and an overview are provided for the teacher. Student prerequisites are: working comfortably with 
manipulativeness, distinguishing basic two-dimensional geometric shapes, making simple patterns, and the 
ability to count past twenty. They must also know how to point and click as well as how to drag lines. Sufficient 
exploration, of an open-ended nature, is recommended to acquaint students with Kid Pix. Another interesting 
recommendation is made to "try training a few Kid Pix 'experts' so they can assist their peers. This will free you 
up to work with other students and will go a long way toward improving students' self esteem” (Apple, 1993, p. 
123). 

Assuming students, aged 8 to 10, have sufficient access to computers and the Kid Pix program, exploration 
would entail use of 5 menu items, 33 submenu items, 13 tools, and 13 palette items, not to mention the 39 colors 
available with the color tool and 600 palette elements. The developmental level of the student, alone, would 
limit so-called "sufficient exploration." Figure 1 itemizes these elements, along with unique number 
designations for each (i.e. M2.1, or M5.3, Tl, or P4), and constitutes the first step in what it is called "software 
mapping". Text has been used to represent graphics that normally appear on the screen. Numbers in parentheses 
represent the elements available when the tool or palette item is chosen. 



Figurel. Generic elements of the desktop presented in Kid Pix 


Detailed explanations about, and illustration of, Kid Pix Tools, Menus, icons, the Gallery, sound recording, and 
the Small Kids Mode are provided in a user's guide and not recreated here (Broderbund, 1991). The generic 
elements identified in Figure 1 add only letter and number designations to better catalogue student actions, or 
behavioral content, that accumulate on the screen (desktop). 

It has been known now to begin the excursion into the curriculum activity as it is "played out" on the screen. The 
following worksheets items are listed in the curriculum text (Apple, 1993, pp. 129-50) with spaces eliminated 
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for brevity and are presented here to illustrate the mapping scheme. A worksheet direction requiring the use of 
the Kid Pix program is presented. Other directions not requiring use of the program are not listed. Page numbers 
are provided for referencing (Apple, 1993). Objects are drawn while menu bar, tool and palette selections are 
mapped and written in parentheses. Use of the asterisk (*) with a number indicates n-repetitions of the tool, 
palette, or menu bar item. The key to letters and numbers used in the following software maps can be found in 
Figure 1. 

In presenting the worksheet (pp. 129-130) for activity one, the students "exploring" Kid Pix displayed three 
different pictorial responses or approaches to the direction, "Build a triangle tower that is five rows high." The 
Figure 2 shows different student's responses. 


Response A 



Response B 



(T3, P3.1, P3.7, *5, T2, P2.1, P2.7, *5, M2.7, return) 


Response C 



(T3.P3.1, *5, T2, P2.1.P2.7, *2) 


Figure 2 . I Spy a T riangle ele ments of the desktop pre s ente d in Kid P ix. 


The software maps of the responses are then juxtaposed to identify the similarity of selections and differences in 
order. All responses are similar in that they use the line tool (T2) and 1st and 7th line and pattern palettes, 
respectively. Response A repeatedly uses the tool-palette combination 27 times; response B, 5 times; and 
response C, 2 times. Clearly repetitive tool use is minimized in responses B and C, but with the rectangle tool 
(T3) and rectangle palettes (P3.1 and P3.7) used to set the unique outer border of their representations of 
triangle. Responses A and B were printed (M2.7) as hard copy. 

In presenting the worksheet for activity two, (p. 135): What Am I ?, with the direction, "Draw a shape using Kid 
Pix Paste your shape in the space below," a student constructed Figure 3 using the pencil (Tl) tool with varying 
sharpness of dots (PI.3, PI.7), then using the stamp (T10) tool to access a lightning bolt (P10.110) and stamped 
three "bolts" (*3) above the umbrella. The truck (Til) tool and palette (PI 1.14) was used to copy (M3.3) and 
paste (M3.4) the next set of three bolts above the umbrella. After printing (M2.7) hard copy the student erased 
the drawing with the eraser (T8) tool and palette (P8.5). Both activities above ask for a drawing but the response 
to activity two illustrates a lengthier response map using additional activity. 
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(Tl, PI.3, PI.7, draw, T10, P10.110, *3, Til, PI 1.14, M3.3, M3.4, M2.7, T8, P8. 

Figure 3. What Am I? 

Worksheets A (p. 141), B (p.143), and C (p.145) in activity three direct students to use the stamp tool to create 
and manipulate patterns. When directed to "Use the stamp tool in Kid Pix to make these patterns. Print the 
patterns. Dog, dog, strawberry, sun (Repeat this pattern four times.)," Figure 4 illustrates stamping two dogs 
and a strawberry ( T10, P10.3, *2, PI 0.4, *1) then using the truck tool (Til) and "grab" palette (PI 1.14) to select 
the drawing, copies and pastes three more sets (M3.3, M3.4, *3). Recognizing that the sun is missing from the 
drawing the student returns to the truck tool, with "grab" palette already highlighted, selects the 3 element sets, 
cuts three sets from the desktop and adds the sun element (error recognition. Til, select, M3.2, T10, P10.13, 
*11). The truck tool copying and pasting sequence is repeated (Tl 1, PI 1.14, select, M3.3, M3.4, *3). 

XX# # 

XX# # 

XX## 

XX# # 

(T10, P10.3, *2, P10.4, *1, Til, PI 1.14, select, M3.3, M3.4, *3, [errorrecognition. Til, select, M3.2, T10, 

P10.13, *1] Til, PI 1.14, select, M3.3, M3.4, *3) 

Figure 4. Worksheet A 

Worksheet B directs students to create a pattern (T10, P10.21, *2, P10.24, Tl, Pl.l, PI.7, draw), print it out 
(M2.7, return), add sound to the pattern-using "Record" under the Goodies menu-, to speak into the microphone, 
and describe the pattern (M4.5, record, return, M4.6). Figure 5 illustrates the output and map. Our fourth activity 
direction is "Use the stamps in Kid Pix to create a pattern of your own. Print it out. Paste it in the space below. 
Add sound to your pattern. Use 'Record' under the Goodies menu. Speak into the microphone and describe your 
pattern." (p. 143). 




(T10, P10.21, *2, P10.24, Tl, Pl.l, P1.7, draw, M2.7, return, M4.5, record, return, M4.6) 


Figure 5. Worksheet B 

Worksheet C is similar to A in creating the elements (T10, P10.1, P10.14, P10.15, P10.28, P10.29, P10.42), 
copying and pasting the patterns (Til, P11.14, select, M3.3, M3.4, *3), then uses the eraser tool to remove 
elements from the pattern (T8, P8.1, select). Finally the pattern is printed out (M2.7) as hard copy and given to a 
partner to fill in the missing parts. The Figure 6 shows the implication of the following activity directions 
(p.145): 

1. Use the stamps in Kid Pix to make a pattern. Be sure to repeat the pattern at least two times. 
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2. Leave one or two blank spots in your pattern, or use the eraser tool to remove one or two stamps from your 
pattern. 

3. Print out your incomplete pattern and give it to your partner. Ask your partner to fill in the missing parts. 


T«*i 


6 




(T10, P10.1, P10.14, P10.15, P10.28, P10.29, P10.42, Til, P11.14, select, M3.3, M3.4, *3, T8, P8.1, select, 

M2.7) 


Figure 6. Worksheet C 

While the drawn images may appear similar in presentation, the construction using menu bar, tool, and palette 
options are again highlighted through use of the software map. 


In the final activity (Venn Diagrams, pp. 149-150) students use Kid Pix to draw two large overlapping circles on 
the screen (T4, P4.1, draw). Then the student must enter names of two groups of objects above the left and right 
circles, respectively (T9, P9.6, P9.15, *2, P9.4, P9.18, P9.5, P9.4). Then the student stamps images-in the 
correct circle, as well as the overlapping segments (T10, P10.4, P10.5, P10.54, P10.59, P10.84, P10.109) .The 
result is printed as hard copy (M2.7). Figure 7 shows the implication of the following directions (pp. 149-150): 

1. Use Kid Pix to draw two large circles on the screen. The circle tool will make it easy. Draw the 

circles so they come together in the middle like the ones you see below. 

2. Name the group of objects that you will stamp in the left circle. Write that name above the left circle. 

3. Name the group of objects that you will stamp in the right circle. Write that name above the right circle. 

4. Add at least five stamps to your drawing. Put each stamp in the correct circle. If it fits in both circles, put it 
where the circles come together. 


Food Red 



(T4, P4.1, draw, T9, P9.6, P9.15, *2, P9.4, P9.18, P9.5, P9.4, T10, P10.4, P10.5, P10.54, P10.59, P10.84, 

P10.109, M2.7) 


Figure 7. Venn Diagrams 

IMPLICATIONS 

Software mapping is a simple but dynamic method that facilitates access to general software program parameters 
and engenders knowledge of powerful but embedded and "deep-rooted" options allowing for increased inter¬ 
rater reliability in documenting or replicating richly constructed student responses. Rich and diverse student 
constructions can only be mediated by rich and diverse curriculum activities. Designing a sequenced set of 
student activities which use program menus, submenus, and palettes is constructing probable maps associated 
with each activity assures divergence in thinking on the part of the student. Since the software is rich with 
options, there will assuredly be many different responses that can be categorized. 

Mapping provides a somewhat "standardized" approach for increasing teacher understanding of full-program 
options present in any software package although the constructed maps will vary in form, content, and screen 
representation. Teachers should not expect software to complement their transmission views of teaching but 
should be ready for expressions and representations of a unique nature. 

Software, as we perceive it, enables us to see that the "...use made of a large collection of information depends 
on how it is organized. One of the values of computers is that they are able, on command, to reorganize 
information in a variety of ways, thereby enabling people to make more and better uses of the 
collection”(AAAS, 1993, p.202). However, software itself, mirrors or models information organization. But 
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unfortunately this exploration of software deep structure, or concept mapping is compromised by strict reliance 
on mandates such as "Read and follow step by-step instructions in a calculator or computer manual when 
learning new procedures” (AAAS, 1993, p. 290). 

It is proposed that if exploration, or free play, as it has been called, is insufficient to "map out the rich terrain" of 
software then it is up to the curriculum activity designer to ensure that activities are diverse rather than 
redundant in its use of the desktop comprised of menus, submenus and tools. Designed curriculum activities 
must be attended to carefully as they are "played out" on the screen to ensure existence of an adequate "window" 
into the psychomotor and cognitive functioning of the learner. The presence of a "window" into the mind of the 
student allows the teacher to form judgments as to how responses are constructed. Since the software is rich with 
options, there will assuredly be many different responses that can be categorized as appropriate for such 
worksheet activities. Authors of activities must ensure a "match" with the curriculum. The term "match" is used 
to ensure that curriculum activity is enhanced by the software, rather than being a burden, and does not mean 
getting an answer, constructed in the mind of the teacher, as the standard against which student responses are 
compared. The scores for software mapping might be derived from a total tally of menu, tool and palette 
options selected by a student. Software mapping can be used for analyzing the students' interaction with the 
software via process schema. Each interaction will be scored as one point in this prosess. A special note of 
gratitude must be extended to Dr. Albert P. Nous for his invaluable assistance and support during the pursuit of 
this method. 

Software maps provide a unique opportunity to research how students construct new knowledge. This medium 
also enables researchers to record students' decision-making as they selectively integrate information into their 
conceptual framework. It stresses the key concepts and propositions that underlie what we label as 'cognition 
patterns' during software mapping. These maps show us how children's metacognition is organized and also how 
they organize information into a meaningful structure. This technique is also a helpful guide for (1) instructional 
software designers to improve and produce high quality software programs, (2) teachers and instructors to 
assess the performance of students as they work their way through their programs, and, (4) representing 
students' interactions with software. 

While computer applications and the underlying assumptions of the role of computers in assessment open up 
doors of opportunity for the development of innovative computer based tools, they also raise serious issues. 
Some of the key issues relate to validity, gender, equity, instructional delivery, the mode of user interface, and 
responsibility to the public (Kumar, 2005). Therefore more research is needed in the development and 
evaluation of computer-based assessment applications that are valid on a large scale. Due to progress in 
computer technology, using virtual reality to stimulate hands-on assessment tasks may be useful for designing 
more effective computer-based performance assessment applications in terms of less obtrusive user interface and 
an increased sense of realism. 
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